You might be forgiven for thinking that the most important algorithm of the next decade will be graph neural networks. Or perhaps Bayesian inference will come to the fore, now that it has a Gartner-friendly name. Least squares will probably do more lifting than both, frankly, and let's not forget voting -- assuming anyone cares about the results of that (though I doubt the LinkedIn poll will prove to be the most important mechanism of our time). I invite you to consider another candidate. Let's play "What is this algorithm and where are the articles about it on Towards Data Science?" Its misfortune is the double one that it is not the product of human design and that the people guided by it usually do not know why they are made to do what they do.

In recent years, trends towards studying object-based games have gained momentum in the fields of artificial intelligence, cognitive science, psychology, and neuroscience. In artificial intelligence, interactive physical games are now a common testbed for reinforcement learning (François-Lavet et al., 2018; Leike et al., 2017; Mnih et al., 2013; Sutton and Barto, 2018) and object representations are of particular interest for sample efficient and generalizable AI (Battaglia et al., 2018; Greff et al., 2020; van Steenkiste et al., 2019). In cognitive science and psychology, object-based games are used to study a variety of cognitive capacities, such as planning, intuitive physics, and intuitive psychology (Chabris, 2017; Ullman et al., 2017). Developmental psychologists also use object-based visual stimuli to probe questions about object-oriented reasoning in infants and young animals (Spelke and Kinzler, 2007; Wood et al., 2020). In neuroscience, object-based computer games have recently been used to study decision-making and physical reasoning in both human and non-human primates (Fischer et al., 2016; McDonald et al., 2019; Rajalingham et al., 2021; Yoo et al., 2020). Furthermore, a growing number of researchers are studying tasks using a combination of approaches from these fields.

The paradigm of pretrained deep learning models has recently emerged in artificial intelligence practice, allowing deployment in numerous societal settings with limited computational resources, but also embedding biases and enabling unintended negative uses. In this paper, we treat pretrained models as objects of study and discuss the ethical impacts of their sociological position. We discuss how pretrained models are developed and compared under the common task framework, but that this may make self-regulation inadequate. Further how pretrained models may have a performative effect on society that exacerbates biases. We then discuss how pretrained models move through actor networks as a kind of computationally immutable mobile, but that users also act as agents of technological change by reinterpreting them via fine-tuning and transfer. We further discuss how users may use pretrained models in malicious ways, drawing a novel connection between the responsible innovation and user-centered innovation literatures. We close by discussing how this sociological understanding of pretrained models can inform AI governance frameworks for fairness, accountability, and transparency.

In this work we propose a novel task framework under which a variety of physical reasoning puzzles can be constructed using very simple rules. Under sparse reward settings, most of these tasks can be very challenging for a reinforcement learning agent to learn. We build several simple environments with this task framework in Mujoco and OpenAI gym and attempt to solve them. We are able to solve the environments by designing curricula to guide the agent in learning and using imitation learning methods to transfer knowledge from a simpler environment. This is only a first step for the task framework, and further research on how to solve the harder tasks and transfer knowledge between tasks is needed.